Skip to content

test: add NOTEEVENTS demo data and enable multimodal mortality test#967

Open
haoyu-haoyu wants to merge 1 commit intosunlabuiuc:masterfrom
haoyu-haoyu:fix/noteevents-test-data
Open

test: add NOTEEVENTS demo data and enable multimodal mortality test#967
haoyu-haoyu wants to merge 1 commit intosunlabuiuc:masterfrom
haoyu-haoyu:fix/noteevents-test-data

Conversation

@haoyu-haoyu
Copy link
Copy Markdown
Contributor

Summary

  • Add synthetic NOTEEVENTS.csv.gz to test-resources/core/mimic3demo/
  • Remove @unittest.skip from multimodal mortality prediction test
  • The NOTEEVENTS preprocessor already exists in MIMIC3Dataset — only test data was missing

What was done

The preprocess_noteevents() function in mimic3.py handles null charttime values correctly. The YAML config for NOTEEVENTS is properly set up. The only gap was missing test data, which caused the multimodal test to be skipped.

Synthetic NOTEEVENTS data

  • Uses SUBJECT_ID and HADM_ID values from existing demo admissions
  • Contains clinical notes with categories: Nursing, Physician, Discharge summary
  • CHARTTIME values are within admission windows
  • All ISERROR = 0

Test plan

  • CI passes
  • test_multimodal_mortality_prediction_mimic3_set_task runs (was previously skipped)

Closes #512

Add synthetic NOTEEVENTS.csv.gz to mimic3demo test resources and
remove the skip decorator from the multimodal mortality prediction
test. The noteevents preprocessor already exists in MIMIC3Dataset.

Changes to the multimodal test:
- Load a dedicated dataset with the noteevents table included
- Replace deprecated .samples attribute with len()/indexing
- Use int() on mortality label for tensor-safe comparison
- Assert at least one sample has non-empty clinical notes

Closes sunlabuiuc#512

Assisted-by: Claude Code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

parser for table NOTEEVENTS is not implemented yet.

1 participant